Logical Methods in Computer Science 
Vol. 4 (4:16) 2008, pp. 1-35 
www.lmcs-online.org 



Submitted Sep. 3, 2007 
Published Dec. 24, 2008 



ON THE EXPRESSIVE POWER OF 
2-STACK VISIBLY PUSHDOWN AUTOMATA 



BENEDIKT BOLLIG 



LSV, ENS Cachan, CNRS — 61, avenue du President Wilson, 94235 Cachan Cedex, France 
e-mail address: bollig@lsv.ens-cachan.fr 



Abstract. Visibly pushdown automata are input-driven pushdown automata that rec- 
ognize some non-regular context-free languages while preserving the nice closure and de- 
cidability properties of finite automata. Visibly pushdown automata with multiple stacks 
have been considered recently by La Torre, Madhusudan, and Parlato, who exploit the 
concept of visibility further to obtain a rich automata class that can even express prop- 
erties beyond the class of context-free languages. At the same time, their automata are 
closed under boolean operations, have a decidable emptiness and inclusion problem, and 
enjoy a logical characterization in terms of a monadic second-order logic over words with 
an additional nesting structure. These results require a restricted version of visibly push- 
down automata with multiple stacks whose behavior can be split up into a fixed number 
of phases. 

In this paper, we consider 2-stack visibly pushdown automata (i.e., visibly pushdown 
automata with two stacks) in their unrestricted form. We show that they are expressively 
equivalent to the existential fragment of monadic second-order logic. Furthermore, it 
turns out that monadic second-order quantifier alternation forms an infinite hierarchy wrt. 
words with multiple nestings. Combining these results, we conclude that 2-stack visibly 
pushdown automata are not closed under complementation. 

Finally, we discuss the expressive power of Biichi 2-stack visibly pushdown automata 
running on infinite (nested) words. Extending the logic by an infinity quantifier, we can 
likewise establish equivalence to existential monadic second-order logic. 



The notion of a regular word language has ever played an important role in computer 
science, as it constitutes a robust concept that enjoys manifold representations in terms of 
finite automata, regular expressions, monadic second-order logic, etc. Generalizing regular 
languages towards richer classes and more expressive formalisms is often accompanied by 
the loss of robustness and decidability properties. It is, for example, well-known that 
the class of context-free languages, represented by pushdown automata, is not closed under 
complementation and that universality, equivalence, and inclusion are undecidable problems 
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Visibly pushdown languages have been introduced by Alur and Madhusudan to over- 
come this deficiency while subsuming many interesting and useful context-free properties [1]. 
Visibly pushdown languages are represented by special pushdown automata whose stack op- 
erations are driven by the input. More precisely, the underlying alphabet of possible actions 
is partitioned into (1) call, (2) return, and (3) internal actions, which, when reading an ac- 
tion, indicates if (1) a stack symbol is pushed on the stack, (2) a stack symbol is read and 
popped from the stack, or (3) the stack is not touched at all, respectively. Such a partition 
gives rise to a call-return alphabet. Though this limits the expressive power of pushdown 
automata, the such defined class of visibly pushdown languages is rich enough to model var- 
ious interesting non-regular properties for program analysis. Even more, this class preserves 
some important closure properties of regular languages, such as the closure under boolean 
operations, and it exhibits decidable problems, such as inclusion, that are undecidable in the 
context of general pushdown automata. Last but not least, the visibly pushdown languages 
are captured by a monadic second-order logic that makes use of a binary nesting predicate. 
Such a logic is suitable in the context of visibility, as the nesting structure of a word is 
uniquely determined, regardless of a particular run of the pushdown automaton. The logi- 
cal characterization smoothly extends the classical theory of regular languages [7,10]. For 
context-free languages, quantification over matchings, which are not implicitly given when 
we do not have visibility, is necessary to obtain a logical characterization [15]. 

Visibly pushdown automata with multiple stacks have been considered recently and 
independently by La Torre, Madhusudan, and Parlato [13], as well as Carotenuto, Murano, 
and Peron [8] . The aim of these papers is to exploit the concept of visibility further to obtain 
even richer classes of non-regular languages while preserving important closure properties 
and decidability of verification-related problems such as emptiness and inclusion. 

In [13], the authors consider visibly pushdown automata with arbitrarily many stacks. 
To retain the nice properties of visibly pushdown automata with only one stack, the idea 
is to restrict the domain, i.e., the possible inputs, to those words that can be divided into 
at most k phases for a predefined k. In every phase, pop actions correspond to one and 
the same stack. These restricted visibly pushdown automata have a decidable emptiness 
problem, which is shown by a reduction to the emptiness problem for finite tree automata, 
and are closed under union, intersection, and complementation (wrt. the domain of fc-phase 
words). Moreover, a word language is recognizable if, and only if, it can be defined in 
monadic second-order logic where the usual logic over words is expanded by a matching 
predicate that matches a push with its corresponding pop event. As mentioned above, such 
a matching is unique wrt. the underlying call-return alphabet. The only negative result in 
this regard is that multi-stack visibly pushdown automata cannot be determinized. 

The paper [8] considers visibly pushdown automata with two stacks and call-return 
alphabets that appear more general than those of [13]: Any stack is associated with a 
partition of one and the same alphabet into call, return, and local transitions so that an 
action might be both a call action for the first stack and, at the same time, a return action 
for the second. In this way, both stacks can be worked on simultaneously. Note that, if we 
restrict to the alphabets of [13] where the stack alphabets are disjoint, the models from [8] 
and [13] coincide. Carotenuto et al. show that the emptiness problem of their model is 
undecidable. Their approach to gain decidability is to exclude simultaneous pop operations 
by introducing an ordering constraint on stacks, which is inspired by [6] (see also [3]). More 
precisely, a pop operation on the second stack is only possible if the first stack is empty. 
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Under these restrictions, the emptiness problem turns out to be decidable in polynomial 
time (note that the number of stacks is fixed) Q 

In this paper, we consider 2-stack visibly pushdown automata (i.e., visibly pushdown 
automata with two stacks) where each action is exclusive to one of the stacks, unless we deal 
with an internal action, which does not affect the stacks at all. Thus, we adopt the model 
of [13], though we have to restrict to two stacks for our main results. One of these results 
states that the corresponding language class is precisely characterized by the existential 
fragment of monadic second-order logic where a first-order kernel is preceded by a block 
of existentially quantified second-order variables. In a second step, we show that the full 
monadic second-order logic is strictly more expressive than its existential fragment so that 
we conclude that 2-stack visibly pushdown automata are not closed under complementation. 
Note that our model has an undecidable emptiness problem, as can be easily seen. 

The key technique in our proofs is to consider words over call-return alphabets as 
relational structures, called nested words [2]. Nested words augment ordinary words with 
a nesting relation that, as the logical atomic predicate mentioned above, relates push with 
corresponding pop events. More precisely, we consider a nested word to be a graph whose 
nodes are labeled with actions and are related in terms of a matching and an immediate- 
predecessor relation. We thus deal with structures of bounded degree: every node has at 
most two incoming edges (one from the immediate predecessor and one from a push event 
if we deal with a pop event operating on the non-empty stack) and, similarly, at most two 
outgoing edges. As there is a one-to-one correspondence between words and their nested 
counterpart, we may consider nested- word automata [2], which are equivalent to visibly 
pushdown automata but operate on the enriched word structures. There have been several 
notions of automata on graphs and partial orders [18, 19] that are similar to nested-word 
automata and have one idea in common: the state that is taken after executing some event 
depends on the states that have been visited in neighboring events. Such defined automata 
may likewise operate on models for concurrent-systems executions such as Mazurkiewicz 
traces [9] and message sequence charts [5]. In the framework of nested- word automata, to 
determine the state after executing a pop operation, we therefore have to consider both the 
state of the immediate-predecessor position and the state that had been reached after the 
execution of the corresponding push event. To obtain a logical characterization of nested- 
word automata over two stacks, we adopt a technique from [5]: for a natural number r, 
we compute a nested- word automaton B T that computes the sphere of radius r around any 
event i, i.e., the restriction of the input word to those events that have distance at most 
r from i. Once we have this automaton, we can apply Hanf's Theorem, which states that 
satisfaction of a given first-order formula depends on the number of these local spheres 
counted up to a threshold that depends on the quantifier-nesting depth of the formula [11]. 
This finally leads us to a logical characterization of 2-stack visibly pushdown automata 
in terms of existential monadic second-order logic. Note that our construction of B r is 
close to the nontrivial technique applied in [5]. In the context of nested words, however, 
the correctness proof is more complicated. The fact that we deal with two stacks only is 
crucial, and the construction fails as soon as a third stack comes into play. 

■*"In [8], the authors argue that 2-stack visibly pushdown automata without restriction are closed under 
complementation, but their proof makes use of the incorrect assumption that these automata are determiniz- 
able. In fact, 2-stack visibly pushdown automata can in general not be determinized [13]. In the present 
paper, we show that 2-stack visibly pushdown automata are actually not closed under complementation. 
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Then, we exploit the concept of nested words to show that full monadic second-order 
logic is more expressive than its existential fragment. This is done by a first-order interpre- 
tation of nested words over two stacks into grids, for which the analogous result has been 
known [17]. 

An extension of Hanf's Theorem has been established to cope with infinite structures [4]. 
This allows us to apply the automaton B r to also obtain a logical characterization of the 
canonical extension of 2-stack visibly pushdown automata towards Biichi automata running 
on infinite words. 

Outline of the paper. In Section [21 we introduce multi-stack visibly pushdown automata, 
running on words, as well as multi-stack nested- word automata, which operate on nested 
words. We establish expressive equivalence of these two models. Section [3] recalls monadic 
second-order logic over relational structures and, in particular, nested words. There, we 
also state Hanf's Theorem, which provides a normal form of first-order definable properties 
in terms of spheres. The construction of the sphere automaton B r , which is, to some 
extent, the core contribution of this paper, is the subject of Section [4.21 By means of this 
automaton, we can show expressive equivalence of 2-stack visibly pushdown automata and 
existential monadic second-order logic (Section l4.ip . Section [5] establishes the gap between 
this fragment and the full logic, from which we conclude that 2-stack visibly pushdown 
automata cannot be complemented in general. By slightly modifying our logic, we obtain, 
in Section [U a characterization of Biichi 2-stack visibly pushdown automata, running on 
infinite words. We conclude with Section [7] stating some related open problems. 



2. Multi-Stack Visibly Pushdown Automata 

The set {0, 1, 2, . . .} of natural numbers is denoted by M, the set {1, 2, . . .} of positive 
natural numbers by JN+. We call any finite set an alphabet. For a set S, we denote by E* 
E + , and E w the sets of finite, nonempty finite, and infinite strings over E, respectivelyo 
The empty string is denoted by s. For a natural number n S M, we let [n] stand for the set 
{1, . . . ,n} (i.e., [0] is the empty set). In this paper, we will identify isomorphic structures 
and we use = to denote isomorphism. 

Let K > 1 be a positive natural number. A (.ff-stack) call-return alphabet is a collection 
££)} sg r.K], £j n f) °f pairwise disjoint alphabets. Intuitively, Ej! contains the actions 
that call the stack s, E* is the set of returns of stack s, and Xj„t is a set of internal actions, 
which do not involve any stack operation. 

We fix K > 1 and a K-stack call-return alphabet S = ({(S®, ££)} se rjr], Ej^). Moreover, 
we set E c = \J sE [ K ] E r = U s e[K] S r> and E = E c U E r U E int . 



2.1. Multi-Stack Visibly Pushdown Automata. 

Definition 2.1. A multi-stack visibly pushdown automaton (Mvpa) over E is a tuple A = 
(Q,T,5,Qi,F) where 

• Q is its finite set of states, 

• Qi Q Q is the set of initial states, 

• F C Q is the set of final states, 

2 From now on, to avoid confusion with nested words, we use the term "string" rather than "word" if we 
deal with elements from E* U . 
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• T is the finite stack alphabet containing a special symbol _L that will represent the empty 
stack, and 

• 5 provides the transitions in terms of a triple (S c , S r , <5j n j) with 

5 C C Q x E c x (r\{±» x Q, 

5 r C Q x E r x T x Q, and 

<W C Q x Sj ni x Q . 

A 2-stack visibly pushdown automaton (2vpa) is an Mvpa that is defined over a 2-stack 
alphabet (i.e., K = 2). 

A transition (q, a, A, q') G 5 C , say with a G EJ, is a push transition meaning that, being 
in state q, the automaton can read a, push the symbol A G T\ {_L} onto the s-th stack, and 
go over to state q'. A transition (q, a, A, q') G 8 r , say with a G E£, allows us to pop ^4 7^ _L 
from the s-th stack when reading a, while the control changes from state q to state q' . If, 
however, A = _L, then the stack is not touched, i.e., _L is never popped. Finally, a transition 
(q,a,q') G <5j„f is applied when reading internal actions a € Ej nt . They do not involve any 
stack operation and, actually, do not even allow us to read from the stack. 

Let us formalize the behavior of the Mvpa A. A stack contents is a nonempty finite 
sequence from Cont = (T \ {-L})* • {-L}. The leftmost symbol is thus the top symbol of the 
stack contents. A configuration of A consists of a state and a stack contents for every stack. 
Hence, it is an element of Q x Cont^ K \ Consider a string w = a% . . . a n G E + . A run of 
A on w is a sequence p = (qo, o~q, . . . , a^) . . . (q n , a^, . . . , cr^) G (Q x Cont^) + such that 
Qo G Qi, <Jq = _!_ for each stack s G [K], and, for all i G {1, . . . , n}, the following hold: 
[Push]: If cij G for s G [iT], then there is a stack symbol A G T \ {_L} such that 

(qi-i, 04, A, q^ G 5 C , af = A- cr|_ l5 and erf = cr?^ for every s' G [if] \ {s}. 
[Pop]: If ai G for s G [if], then there is a stack symbol A G T such that ((/i_i, aj, A, G 
5 r , af = for every s' G [if] \ {s}, and either A / 1 and o~f_ l = j4 ■ af , or A = _L and 

[Internal]: If a« G S irit , then Oj, G <5 in t, and cr| = <r|_ 1 for every s G [K]. 

The run p is accepting if q n G F. A string to G S + is accepted by A if there is an accepting 
run of A on w. The set of accepted strings forms the (string) language of A, which is a 
subset of S + and denoted by L(^4)0 

Example 2.2. There is no Mvpa that recognizes the context-sensitive language {a n b n c n \ 
n > 1}, no matter which call-return alphabet we chose. Note that, however, with the more 
general notion of a call-return alphabet from [8] , it is possible to recognize this language by 
means of two stacks. Now consider the 2-stack call-return alphabet E given by = {a}, 
E r x = {a}, E^ = {&}, S r 2 = {&}, and E mL = 0. The language L = {{ab) n a n+1 b n+1 \ n > 1} 
can be recognized by some 2vpa over E, even by the restricted model of 2-phase 2vpa 
from [13], as every word from L can be split into at most two return phases. In the following, 
we define a 2vpa A = ({qo, ■ ■ ■ , q^}, {$, -L}, 5, {qo}, {qo}) over E such that L(A) = L + , which 
is no longer divisible into a bounded number of return phases. The transition relation 5 is 



'To simplify the presentation, the empty word e is excluded from the domain. 
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given as follows (a graphical illustration is provided in Figured]): 

5 C : (q ,a,$,q 2 ) 5 r : (q 3 ,a,$,q 3 ) 

(q 2 ,b,$,q 1 ) (q 3 ,a,±,q 4 ) 

(q 1 ,a,$,q 2 ) (q^b,$,q 4 ) 

(q 2 ,b,$,q 3 ) (q 4 ,b,±,q ) 

The idea is that the finite-state control ensures that an input word matches the regular 
expression ((ab) + a + b + ) + . To guarantee that, in any iteration, the number of o is by one 
less than the number of a, any push action a stores a stack symbol $ in stack 1, which can 
then be removed by the corresponding pop action a unless the symbol _L is discovered. We 
do the same for b and b on stack 2. 




Figure 1: A 2VPA 



2.2. Nested Words and Multi-Stack Nested- Word Automata. We will now see how 
strings over symbols from the call-return alphabet X can be represented by relational struc- 
tures. Basically, to a string, we add a binary predicate that combines push with correspond- 
ing pop events. Let s G [K]. A string w G X* is called s-well formed if it is generated by 
the context-free grammar 

A ::= aAb \ AA \ e \ c 
where a G Xj!, b G X*, and c G X \ (X* U X*). 

Definition 2.3. A nested word over X is a structure ([n], <, /i, A) where n G JN+ (we call 
the elements from [n] positions, nodes, or events), < = {{i,i + 1) | i G [n — 1]}, A : [n] — > X, 
and n = Use[_ft:] ^ s ^= M x M where, for every s G [K] and (i, j) G [n] x [n], (i, j) G /i s iff 
z < j, X(i) G X*, A(j) G X£, and A(i + 1) . . . A(j - 1) is s-well formed. 

The set of nested words over X is denoted by NW(X). 

Figure [2] depicts a nested word over a 2-stack call-return alphabet. Throughout the 
paper, we take advantage of the fact that nested words over a 2-stack call-return alphabet 
can be written as a string with one type of stack edges above the string and the other below 
the string, where the first type concerns the first stack and the other type concerns the 
second stack. In the 2-stack case, the edges do not intersect. 

Note that a nested word needs not be well-matched. It might have pending calls, i.e., 
calls without matching return, as well as pending returns, i.e., returns that do not have 
a matching call. Therefore, the relations fi and its inverse can be seen as partial 
maps [n] [n], in the obvious manner. Moreover, observe that, given nested words 
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W = ([n],<,/4,A) and W = ([n'], <•',//, A'), n = ra' A A = A' implies VF = W. It is 
therefore justified to represent W as the string string(W) := A(l)...A(n) £ S + . This 
naturally extends to sets C of nested words and we set string(£) := {string(W) | W € £}. 
Vice versa, given a string w S £ + , there is precisely one nested word W over £ such 
that string(W) = w. This unique nested word is denoted nested(w). For L C £ + , we let 
nested(L) := {nested(u>) | w £ L}. 

Example 2.4. Consider the 2-stack call-return alphabet £ from Example 12.21 which was 
given by = {a}, £* = {a}, = {b}, Y% = {6}, and = 0. Figure [2] depicts a nested 
word iy = ([n], <, /i, A) over £ with n = 10. The straight arrows represent <, the curved 
arrows capture \i (those above the horizontal correspond to the first stack). For example, 
(2,9) € (J,. Thus, fi(2) and /i _1 (9) are defined, whereas both \i 1 (7) and fi 1 (10) are not. In 
terms of visibly pushdown automata, this means that positions 7 and 10 are employed when 
the first/second stack is empty, respectively. Observe that W = nested(a b a baa abb b) and 
string(T / F) = ababaaabbb. 




Figure 2: A nested word 

We now turn to an automata model that is suited to nested words and, to some extent, 
is equivalent to Mvpa. Our model is an extension of nested- word automata for one stack, 
which has been considered in [2], to multiple stacks. We also extend the model of [2] by 
calling states. If the state that is reached after executing some action a is a calling state, 
then the corresponding run is accepting only if this a is a call with a matching return (i.e., 
it is not pending). We will later see that this concept does not increase the expressive power 
of our automata but turns out to be a convenient tool when we translate logical formulas 
into automata. 

Definition 2.5. A generalized multi-stack nested-word automaton (generalized Mnwa) over 
S is a tuple B = (Q, 5, Qi, F, C) where 

• Q is the finite set of states, 

• Qi Q Q is the set of initial states, 

• F C Q is the set of final states, 

• C C Q is a set of calling states, and 

• 5 is a pair (Si, 62} of relations 5\ C Q x S x Q and £2 Q Q x Q X S r x Q, which contain 
the transitions. 

We call B a multi-stack nested-word automaton (Mnwa) if C = 0. 

A (generalized) 2-stack nested-word automaton ((generalized) 2nwa) is a (generalized, 
respectively) Mnwa that is defined over a 2-stack alphabet (i.e., K = 2). 
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Intuitively, 5\ contains all the local and push transitions, as well as all the pop transi- 
tions that act on an empty stack (i.e., in terms of nested words and nested- word automata, 
those transitions that perform an action from E r that is not matched by a correspond- 
ing calling action). A run of B on a nested word W = ([n], <,p, A) over E is a mapping 
p : [n] — > Q such that (q, A(l), p(l)) € <5i for some q € Qj, and, for all i € {2, . . . ,n}, we 
have 



language of B, denoted by £(£>), is the set of nested words from NW(E) that allow for an 
accepting run of B. 

Recall that there is a one-to-one correspondence between strings and nested words. We 
let therefore C(A) with A an Mvpa stand for the set nested (L (*4)). 

Example 2.6. Consider again the 2-stack call-return alphabet E given by S^, = {a}, 
E* = {a}, Y? c = {5}, = {&}, and Ej„t = 0. In Example 12.21 we have seen that, for 
L = {(ab) n aP' +1 V l \ n > 1}, the iteration L + is the language of some 2VPA over E. We can 
also specify a 2NWA B = ({qo, ■ ■ ■ , q^}, 5, {qo}, {qo}, 0) over E such that C(B) = nested(L + ). 
Note that C{B) will contain, for example, the nested word that is depicted in Figure (2) The 
transition relation 5 is given as follows: 



Similarly to Example 12.21 t ne finite-state control will ensure the general regular structure 
of a word without explicit "counting" . This counting is then implicitly done by the relation 
62, which requires a matching call for a return. A graphical description of B is given in 
Figure O Hereby, a return transition with an adjoining set of states indicates that one state 
of this set must have been reached right after executing the corresponding call (in particular, 
the return must not be pending), whereas the remaining return transitions, (53, a, §4) and 
(qi,b,qo), apply only to pending returns. 




<5i : (go, a,q 2 ) 



(q 2 ,q3,a, q 3 ) 
fe,94,M4) 
(9i,94,&,g4) 



(92,5,9i) 
(9i,a,92) 
(92,5,93) 
(93,a,94) 
(94, 5, q ) 



b 




a 



b 



b 



{91, 93}, 5 C@) 



a 



Figure 3: A 2nwa 
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A general technique for a reduction from Mvpa to Mnwa and vice versa can be found 
below (Lemma 12. 8p . 

We can show that the use of calling states does not increase the expressiveness of 
Mnwa. Note that, however, the concept of calling states will turn out to be helpful when 
building the sphere automaton in Section 14.21 



Lemma 2.7. For every generalized Mnwa B over £, there is an Mnwa B' over £ such 
that C(B') = C(B). 

Proof. In the construction of an Mnwa, we exploit the following property of a nested 
word W = ([n], <,fJ>, A): given G p, say, with X(i) G ££, p{i') is defined for all 

i' G {i + 1, . . . , j — 1} satisfying A(i') G E*. Basically, will simulate £>. In addition, 
whenever a calling state is assigned to a position labeled with an element from we will 
set a flag b[s] = 1, which can only be resolved and turn into a final state (b[s] = 0) when a 
matching return position has been found. As any interim call position that concerns stack 
s is matched anyway, the flags b[s] in that interval are set to 2. Thus, while a flag is 1 or 
2, there is still some unmatched calling position. Hence, a final state requires every flag to 
equal 0, which also designates the initial state. 

Let us become more precise and let B = (Q,8,Qi, F,C) be a generalized Mnwa. We 
determine the Mnwa B' = (Q', 5', Q' v F', 0) by Q' = Q x {0, 1, 2}^ , Q\ = Qj X {(0) s6[if] }, 
F f = Fx {{0) se[K] }, and <5' = (tfj, where 

• 5^ is the set of triples ((q, b), a, (q' , b )) G Q' x E x Q' such that (q,a,q') G 5i, q' £ C 
implies a G E c , and, for every s G [K], 



b'\s] 



2 ifb[s]G{l,2} 

1 if b[s] = and a G £* and q' G C 
otherwise 



$2 is the set of quadruples ((», c), (g, b), a, (</', b )) G Q'xQ'xS r xQ' such that (p, g, a, g') G 
d~2, q' $ C, and, for every s G [if], 



b'w 




if c[s] = 1 
otherwise 



In fact, we can show that C(B) = C(B'). 

Note that the flag assignments depend deterministically on the input word and the 
states assigned to the positions. Let W = ([n], <, fj,, A) be a nested word over E. 

Suppose p to be an accepting run of B on W and let p : [n] — > {0,1,2}^ be the 
unique supplement of p according to the flag construction. To verify that (p, p) is indeed an 
accepting run of B' on W, we need to show that p(n)[s] = for all s G [K]. So let s G [K\. 
If there is no i G [re] such that X(i) G E^ and p(i) G C, then we clearly have p(re)[s] = 0, 
as the flag for stack s never changes its value during the run. If the flag changes its value 
from to 1, then this happens at a position i G [re] such that G £j! and p(i) G C . As p 
is an accepting run of B on W, there is j G [n] such that G p. By construction of B', 

p(i)[s] = 1, p(^')[ s ] = 2 for all i' G {i + 1, . . . , j — 1}, and p(j)[s] = 0. Thus, we finally have 
p(n)[s}=0. 

Conversely, let p : [n] — > Q and p : [n] — > {0, 1, 2}^ be mappings such that (p, p) is an 
accepting run of B' on W. Clearly, p is a run of B on W. So let us verify that it is accepting. 
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First, observe that p{n) G F. So suppose i G [n] such that p{i) is a calling state. According 
to the construction of B' , G for some s. Moreover, we have jo(z)[s] = {1,2}. As 
jo(n)[s] = 0, there must be %' < i and j' > i such that A(i') G and G This 

implies that //(z) is indeed defined so that we can conclude that p is an accepting run of B 
on W. □ 

The flag construction from the previous proof is illustrated in Figure 01 where we assume 
a run on the nested word such that every state associated with a symbol from {a, b} is a 
calling state. 

123456789 10 




a — ► b — ► a — ► b — ► a — > a — > a — > 6 — > b — > b 




Figure 4: The flag construction 



Lemma 2.8. Let C C NW(S) be a set of nested words overT,. The following are equivalent: 

(1) There is an Mvpa A over E swc/i that C{A) = C 

(2) There is an Mnwa B over £ suc/i t/iat >C(jB) = L. 

Proof. Given an Mvpa A = (Q, T, S,Qi,F), we define an Mnwa B = 5', Q' /5 F', 0) with 
= £(B) as follows: Q' = Q x T, Q' 7 = Qi x {i_}, F' = F x T, and 5' = (6[,6' 2 ) where 

• ^ is the set of triples ((q, A), a, (q' , A')) G Q' x S x Q' such that (q,a,A',q') G <5 C , 
(q,a,q') G 5 ini , or (g,a, _L,g') G 5 r , and 

• 5' 2 is the set of quadruples ({p,B),(q,A),a,(q' ,A')) G Q' x Q' x T, x Q' such that 
(q,a,B,q') G 5 r . 

The idea is that the stack symbol associated with a transition is incorporated into the state 
of the Mnwa. When an internal or unmatched return action is performed, then we may 
chose an arbitrary stack symbol, as it will not be reconsidered later in the run. 

For the converse direction, let B = (Q,6,Qi, F,$) be an Mnwa. Consider the Mvpa 
A={Q,QU {±}, 5', Q u F) where 5' = (5' c , 5' r , 5' mt ) is given by 

• S' c = {(q,a,q',q r ) | (q,a,q') G <5i D (Q x S c X Q)}, 

• S int = 5i n (Q x E int x Q), and 

• 8' T is the set of tuples (q,a,A,q') GQxY^rXTxQ such that either (q,a,q') G 6% and 
A = _L, or (A, q, a, q') G (5 2 . 

Here, we need to ensure that, when A performs a matched return action, we can access the 
state that B has associated with the corresponding call. To this aim, A just pushes the state 
onto the stack so that it becomes accessible when the corresponding return is executed. It 
is straightforward to show that C(A) = C{B). □ 
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3. Monadic Second-Order Logic and Hanf's Theorem 

3.1. Monadic Second-Order Logic over Relational Structures. We fix supplies of 
first-order variables x,y, . . . and second-order variables X, Y, . . .. Let r be a function-free 
signature. The set MSO(r) of monadic second- order (MSO) formulas over r is given by the 
following grammar: 

ip ::= P(x\, . . . , x m ) | x\ = X2 \ x £ X | -199 | 991 V 992 \ 3x<p \ 3X(p 

Hereby, m > 1, P G r is an m-ary predicate symbol, the Xk and x are first-order variables, 
and X is a second-order variable. Moreover, we will make use of the usual abbreviations 
such as ifi A if2 for -"(-^i V -^2), ¥>i ~~ y V?2 for -1^1 V (f2, etc. Given a r-structure 
21 with universe A, a formula ip(x\, . . . ,x m ,Xi, . . . , X n ) G MSO(r) with free variables in 
{xi, . . . ,x m ,X 1 , . . . ,X n }, (ui, . . . ,u m ) G A m , and (Ui, . . . , U n ) G (2 A ) n , we write, as usual, 
21 |= ip[ui, . . . , u m , Ui, . . . , U n ] if 21 satisfies 99 when assigning (ui, . . . , u m ) to (xi, . . . , x m ) 
and (Ui,...,U n ) to (Xi,...,X n ). 

Let us identify some important fragments of MSO(r). The set FO(r) of first order (FO) 
formulas over r comprises those formulas from MSO(r) that do not contain any second-order 
quantifier. Furthermore, an existential MSO (EMSO) formula is of the form 3X\ . . . 3X n <p 
with ip G FO(t). The corresponding class of formulas is denoted EMSO(r). More generally, 
given m > 1, we denote by S m (r) the set of formulas of the form 3X^X2 ■ ■ ■ 3/\/X m Lp 
where (p G FO(r) and the X^ are blocks of second-order variables, possibly empty or of 
different length. 

We will later make use of the notion of definability relative to a class of structures. 
Let T C MSO(r) be a class of formulas and C,C be sets of r-structures. We say that C is 
T-definable relative to C if there is a sentence (i.e., a formula without any free variables) 
ip G T such that C is the set of r-structures 21 G C such that 21 |= (p. 

3.2. Hanf's Theorem for Nested Words, and Spheres. We will now provide a signa- 
ture that allows us to specify MSO properties of nested words. Let S be a call-return 
alphabet. We define rg to be the signature {A a | a G S} U {<,fJ-} with A a a unary 
and < and /x binary predicate symbols. We write the MSO formula A„(x) as A(x) = a 
and the formula <(xi,X2) as x\ < X2. MSO formulas over rg can be canonically in- 
terpreted over nested words ([n],<,/x, A) G NW(E), as A can be seen as a collection of 
unary relations A a = {i G [n] \ = a} where a G S. Thus, nested words over 
S are actually rg-structures. A sample MSO formula over rg such that S = {a, b} is 
VxVy (A(x) = a A /u(x, y) — > A(y) = b). It expresses that every matching pair with a calling 
a has a 6-labeled return position. Given a sentence ip G MSO(rg), we denote by C(ip) the 

set of nested words over E that satisfy 99, i.e., £(99) = {W G NW(S) | IF |= <p}. 

Over nested words (more generally, structures of bounded degree), FO formulas enjoy 
a normal form in terms of local formulas. A formula <p(x) G FO(rg) with one free variable 
x is said to be local if there is r G IN such that, in every subformula 3yip of 99, ip is of the 
form (d(x, y) < r) A %. Hereby, the formula d(x, y) < r has the expected meaning and can 
be obtained inductively. Informally, the truth of a local formula ip(x) depends only on the 
local neighborhood around x. 

Next, we state Hanf's locality theorem in terms of nested words. It actually applies to 
general classes of structures of bounded degree. 
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Theorem 3.1 (Hanf [11]). Let ip G FO(rg) be a sentence. There is a positive Boolean 
combination ip of formulas of the form 

3 =t xx{x) and3 >1: xx(x) 

where t G IN and x £ FO(rg) is local (with the obvious meaning of the quantifiers 3 =t and 
3 ; note that there might occur different thresholds t in ip) such that, for every nested word 
W G NW(£), we have 

W\=<piffW\=i/)- 
Moreover, ip can be computed effectively and in elementary time. 

For a comprehensive proof of this theorem, see, for example, [16,20]. However, these 
proofs are not effective, whereas the original proof by Hanf is effective. It is crucial to note 
that Hanf 's Theorem applies to the case of nested words as we deal with a class of structures 
of bounded degree (see below for a formal definition). Indeed, there is a uniform bound on 
the degree of nested words. 

Let 21 = (N,<,fj,,X,...) and 21' = (N', <•', //, A', . . .) be tuples such that (N, <,/x,A) 
and (iV', <', //, A') are rg-structures. For i,j G N and i',f G N', we write (i,j) (i',f) if 
X(i) = X'(i'), A(j) = A'(i'), G < implies (*',;') G <', and G /i implies (*',/) G //. 

Theorem 13.11 suggests that, over nested words, the validity of an FO formula in a nested 
word depends on the local neighborhoods of the latter. This leads to the notion of a sphere, 
which will actually play a central role in the remainder of this paper. A sphere of radius 
r G IN includes elements whose distance from a distinguished sphere center is bounded by 
r. Given i, j G N, the distance d%(i,j) of i and j in 21 is the minimal length of a path from 
i to j in the Gaifman graph of (N, <, fx, A). The Gaifman graph of (N, <, \x, A) is defined to 
be the undirected graph (N, Arcs) where (i,j) € ^4rcs iff G < U ^ U < _1 U [16]. 
In particular, we have d%(i, i) = 0. If d<&(i,j) = 1, we also write % j. We write i — >2t J if 
(i, j) G < U fj,. The degree of a rg-structure is said to be bounded by some natural number 
B if the degree of its Gaifman graph is bounded by B. Observe that the degree of a nested 
word is bounded by 3, which is therefore a uniform bound for the class NW(S). 

Let 2$ = (N, < 5/ u, A) be a rg-structure, r G IN", and i G N. The r-sphere of *B around 
i, which we denote by r-Sph(23,i), is basically the substructure of © induced by the new 
universe {j G N \ dss(i,j) < f}, but extended by the constant i as a distinguished element, 
called the sphere center. Given an isomorphism type S of an r-sphere, we let |Q3|s := \{i G 
N | S = r-Sph(25, i)}\ denote the number of points in *B that realize S. For an example, 
consider Figure El showing a nested word W and the 2-sphere of W around i = 10 where 
the sphere center is marked as a rectangle. Note that 2-Sph(H / , 10) = 2-Sph(W / , 14) and 

l^l2-Sph(VK,10) = 2. 

We denote by Spheres r (E) the set of (isomorphism types of) r-spheres that arise from 
nested words over S, i.e., 

Spheres r (S) := {r-Sph(W, i) | W G NW(E) and i is a node of W} . 

Note that Spheres r (S) is finite up to isomorphism, which is crucial for the constructions in 
Section @J 
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Figure 5: A 2-shpere embedded into a nested word 

4. 2- Stack Visibly Pushdown Automata vs. Logic 

In this section, we focus on 2vpa. So let us fix a 2-stack call-return alphabet £ = 
<{(£!,£!), (£2, £2 )}jS . nt) . 

4.1. The Main Result. The key connection between FO logic and 2vpa/2nwa is provided 
by the following proposition, which states the existence of an automaton that computes the 
sphere around any node of a nested word. 

Proposition 4.1. Let r be any natural number. There are a generalized #NWA B r = 
(Q,5,Qi,F,C) over £ and a mapping n : Q — > Spheres r (E) such that 

• C{B r ) = NW(£) (i.e., every nested word admits an accepting run of B r ), and 

• for every nested word W € NW(£), every accepting run p of B r on W, and every node i 
ofW, we have n(p(i)) = r-Sph(W, i). 

Before we turn towards the proof of this statement, we will first show how Proposi- 
tion 14.11 can be used to establish expressive equivalence of 2vpa and EMSO logic. 

Lemma 4.2. Let r,i £ K and let S € Spheres r (Y>) be an r-sphere in some nested word 
over £. There are generalized 2nwa B 1 and B 2 over £ such that L{B X ) = {W € NW(£) | 
\W\ S = t} and C(B 2 ) = {W £ NW(£) | \W\ S > t}. 

Proof. In both cases, we start from the generalized 2nwa B r = (Q,5,Qi , F,C) and the 
mapping rj : Q — > Spheres r (Yi) from Proposition 14.11 For k = 1,2, we obtain B k by 
extending the state space with a counter that, using rj, counts the number of realizations 
of S up to t + 1. The new set of initial states is thus in both cases Qi x {0}. However, the 
set of final states of B 1 is F x {t}, the one of B 2 is F x {t + 1}. □ 
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We are now prepared to state the first main result of this paper. 

Theorem 4.3. Let C C NW(S) be a set of nested words over the 2-stack call-return alphabet 
S. Then, the following are equivalent: 

(1) There is a £vpa A over £ such that C{A) = C 

(2) There is a sentence ip G EMSO(rg) such that £(</?) = C 

Both directions are effective. In particular, the 2vpa that we construct for a given EMSO 
sentence can be computed in elementary time, and its size is elementary in the size of the 
formula. 

Proof. To prove (1) — > (2), one can perform a standard construction of an EMSO for- 
mula from a 2nwa, where the latter can be extracted from the given 2VPA according to 
Lemma 12.81 Basically, the formula "guesses" a possible run on the input word in terms of 
existentially quantified second-order variables and then verifies, in its first-order fragment, 
that we actually deal with a run that is accepting. 

So let us directly prove (2) -> (1) and let ip = 3Xi . . . 3X m ip(X 1 ,. . . , X m ) G EMSO(t £ ) 
be a sentence with ip(Xi, . . . , X m ) G FO(Tg) (we suppose m > 1). We define a new 2-stack 
call-return alphabet 

£ = ({(£* x 2H j £i x 2H) ; (£2 x 2 M )S 2 x 2 H)} jS . nt x 2 H) 

where 2l m l shall denote the powerset of [m]. From ip, we obtain an FO formula -0' over Tg 
by replacing each occurrence of X(x) = a with \/ MG2 i m l X ( x ) = (°> M) and each occurrence 
of x G X k with Vaes, Me2M X ( x ) = ( a , M u W)- We set £ S NW(E) to be the set of 
nested words that satisfy ip' . From Hanf's Theorem (Theorem 13. ip . we know that C is the 
language of a positive Boolean combination of formulas of the form 3 =t x x( x ) an d 3 >l x x( x ) 
where x is local. It is easy to see that the class of nested- word languages that are recognized 
by generalized 2nwa is closed under union and intersection. Thus, the validity of one such 
basic formula can be checked by a generalized 2nwa due to Lemma 14.21 We deduce that 
there is a generalized 2nwa B' over E recognizing C. 

Now, to check whether some nested word from NW(S) satisfies cp, a generalized 2nwa 
B with C{B) = C(ip) will guess an additional labeling for each node in terms of an element 
from 2l m l and then simulate B' . By Lemma 12.71 and Lemma 12.8} we finally obtain a 2VPA 
A such that C(A) = £(ip). □ 

4.2. Proof of Proposition [4TTI We now turn to the proof of Proposition l4.1i In each state, 
the generalized 2nwa B r will guess the current sphere as well as spheres of nodes nearby 
and the current position in these additional spheres. Adding some global information allows 
us to locally check whether all the guesses are correct. The rest of this section is devoted 
to the construction of B r and a corresponding mapping rj to prove Proposition 14.11 

4.2.1. The Construction. Recall that Spheres r (E) denotes the set of all the r-spheres that 
arise from nested words, i.e., Spheres r (S) = {r-Sph(W, i) \ W is a nested word and i is a 
position in W}. An extended r-sphere over S is a tuple E = (N,<, fi, A, 7, a, col) where 
core(E) := (A r , <, /i, A, 7) G Spheres r (T,) (in particular, 7 G N), a G N, and col G [#Col] 
with jfCol = 4 • maxSize(r) 2 + 1 where maxSize(r) is the maximal size of an r-sphere, 
i.e., maxSize(r) = max{|iV| | (N, <, /i, A, i) G Spheres r (S)} . We say that a is the active 
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node of E and col is its color. Strictly speaking, (iV, <, p,, A, 7, a, col) is not a mathematical 
structure, as col does not refer to an element of N. We introduced the function core to 
extract a mathematical structure from an extended sphere, which will allow us to deal with 
notions such as isomorphism. 

Let eSpheres r (Yi) denote the set of all the (isomorphism classes of) extended spheres 
over E. For an extended sphere E = (N, <, p, A, 7, a, col) and an element i € N, we denote 
by E[i] the extended sphere (N, <, /x, A, 7, i, coZ), i.e., the extended sphere that we obtain 
by replacing the active node a with i. 

The idea of the construction of the generalized 2nwa B r is the following: A state £ 
of B r is a set of extended spheres, which reflect the "environment" of a node that £ is 
assigned to. Now suppose that, in a run of B r on a nested word W = ([n], <,p, X), £ is 
assigned to a position i € [n] and contains E = (TV, <,p, A, 7, a, coZ). If the run is accepting, 
this will mean that the environment of i in W looks like the environment of a in E. In 
particular, £ will contain exactly one extended sphere E = (N, <, p, A, 7, a, coZ) such that 7 
and a coincide, meaning that r-Sph(W,i) = (N, <,p, A, 7). This is illustrated in Figure [6] 
depicting a nested word and a step of a run of the sphere automaton for r = 1 on this 
word. States £ and £' are assigned to positions 4 and 5, respectively. Each state is a 
set of extended spheres. For clarity, however, we will neglect colors in the example. The 
sphere center is, as usual, depicted as a rectangle; the active node is marked as a circle. 
Observe that each state contains precisely one extended sphere in which the sphere center 
and the active node are identical. These are E\ € £, and, respectively, E' 2 € £'■ Indeed, 
Ei corresponds to the 1-sphere of the nested word around 4, while E' 2 reflects the 1-sphere 
around 5. 

Of course, B r has to locally guess the environment of a position. But how can we 
ensure that a guess is correct? Obviously, we have to pass a local guess to each neighboring 
position in W. So suppose again that a state £ containing E = (N, <, p, A, 7, a, col) is 
assigned to a node i of W. As a shall correspond to i, we need to ensure that A(a) = \(i) 
(this will be taken care of by item (2) in the definition of the transition relation below). 
Now suppose that a has a <-successor j 6 N, i.e., a < j. Then, we have to guarantee that 
i < re. This is done by simply excluding £ from the set of final states (in Figure [6j neither £ 
nor £' are final states). Moreover, j should correspond to i + 1, which is ensured by passing 
E[j] to the state that will be assigned to i + 1 (see item (7); in Figure [U £' must therefore 
contain E\[j] where j is the <-successor of the active node of E\, and we actually have 
E[ = Ei[j]). On the other hand, if % has a <-successor, then a must have a <-successor 
j as well such that E[j] belongs to the state that will be assigned to i + 1. Observe that 
this rule applies unless ^(7, a) = r, as then i + 1 lies out of the area of responsibility of E 
(see item (5)). Similar requirements have to be considered wrt. potential <-/<-predecessors 
(see (3), (4), and (6)), as well as wrt. the relations // and Jl (see (3')-(7')). One difficulty 
in our construction, however, is to guarantee the lack of an edge. So assume the extended 
sphere E is the one given by Figure [5] with ji as the active node. Let us neglect colors 
for the moment. Suppose furthermore that W is the nested word from Figure [U below 
the sphere. Then, an accepting run p of B r on W will assign to i\ a state that contains 
E (modulo some coloring). Moreover, the state assigned to i will contain E[j], where the 
sphere center and the active node coincide. We observe that, in E, the node ji is maximal. 
In particular, there is no //-edge between j\ and jY This should be reflected in W. A first 
idea to guarantee this might be to just prevent pfo) from containing the extended sphere 
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Figure 6: A step of the sphere automaton 



E[j2\ (note that (ii,«2) G m). This is, however, too restrictive. Actually, (r-Sph(PF, i), 12) 
and E[j2\ are isomorphic (neglecting the coloring of E) so that p{i2) must contain 22 [72]. 
The solution is already present in terms of the coloring of extended spheres. More precisely, 
p{t2) is allowed to carry 2? [72] as soon as it has a color that is different from the color of the 
extended sphere E\j\] assigned to i%. Roughly speaking, there might be isomorphic spheres 
in W that are overlapping. To consider them simultaneously, they are thus equipped with 
distinct colors. 

The construction we obtain following the above ideas indeed allows us to infer, from an 
accepting run assigning a state £ to a node i, the r-sphere around i. As mentioned above, 
we simply consider the (unique up to isomorphism) extended sphere (TV, <£,//, A, 7, a, col) 
contained in £ such that 7 = 0. Then, (TV, <C, fj,, A, 7) is indeed the sphere of interest (recall 
that, in Figure [6l these are E\ for £ and E' 2 for £' if we ignore active nodes and colors). 

It is not obvious that the above ideas really do work, all the less as the construction 
will apply to nested words over two stacks, but no longer to nested words over more than 
two stacks. After all, the key argument will be provided by Proposition 14.51 stating an 
important property of nested words over two stacks. Intuitively, it states the following: 
Suppose that, in a nested word, there is an acyclic path from a node i to another node i', 
and suppose this path is of a certain type w (recording the labelings and edges seen in the 
path). Then, applying the same path several times will never lead back to i. This is finally 
the reason why a cycle in an extended sphere that occurs in a run on a nested word W is 
in fact simulated by W. 
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Let us formally construct the generalized 2NWA B r = (Q, 6, Qi,F, C). An element of Q 
is a subset £ of eSpheres r (Y,) such that either £ = 0, which will be the only initial state, or 
the following conditions are satisfied: 

(a) there is a unique extended sphere (N, < , fj,, A, 7, a, col) G £ such that 7 = 
(we set core(£) := (N, <, fx, X, 7)) 

(b) there is a € X such that, for every (iV, <,fi,X, 7, a, coZ) G £ , A(a) = a 
(so that we can assign a unique label a to £, denoted by label (£)) 

(c) for every two elements E = (TV, <, /i, A, 7, a, col) and I?' = (TV', <•', //, A', 7', a', col') 
from £ , if core(E) = core(E') and coZ = col', then a = a' 

So let us turn to the transition relation 5 = (61,62): 

• For £, £' G Q and a G S, we let (£, a, £') G Si if £' 7^ and the following hold: 

(1) for all (A, < , /i, A, 7, a, col) G £', a g" dom(/n _1 ) (i.e., / u~ 1 (a) is not defined) 

(2) label{£') = a 

(3) for all E = (TV, < , /i, A, 7, a, coZ) G 5 and i G N, 

E[i] G £' => (a,t) G < 

(4) for all £ = (/V, <, fi, A, 7, a, coZ) G 

£ 7^ A -i3z : (i, a) G < ==>- cZe(7, a) = r 

(5) for all E = (N, <,fi, A, 7, a, col) G £, 

Si : (a, i) G < ==> ^(7, a) = r 

(6) for all 75 = (A, <,fi, A, 7, a, col) G £' and i G TV, 

(i, a) g < ==> E[i] G 5 

(7) for all E = (A, <,fi, A, 7, a, coZ) G £ and i G A, 

(a, i) G < => G £' 

• For £ c , £,£' e Q and a G S r , we let (£ c , £, a, £') G (52 if £ c , £, £' / and (2)-(7) as above 
hold as well as the following: 

(3') for all E = (A, < , fj,, A, 7, a, coZ) G £ c and i G A, 

£7[i] G £' => (a,i) G n 
(4') for all £ = (A, <,fi, A, 7, a, col) G £', 

a ^ dom(/i _1 ) =>■ 0^(7, a) =r 
(5') for all E = (A, <,(jl, A, 7, a, coZ) G £ c , 

a G" dom(/i) ==> c?£;(7,a)=r 

(6') for all E = (A, <,fi, A, 7, a, col) G £', 

aedom^ 1 ) =► E[n- 1 (a)\ G £ c 
(7') for all £ = (TV, <, //, A, 7, a, coZ) G £ c , 

a G dom(/x) ==>■ E\p(a)] G £' 

As already mentioned, the only initial state of B r is the empty set, i.e., Qi = {0}. Moreover, 
£ G Q is a final state if, for every extended sphere (N, <, fi, X,j,a, col) G £, both a G" 
dom(//) and there is no i G A such that (a, i) G <. Finally, £ is contained in C, the set of 
calling states, if there is (A, <, /x, A, 7, a, coZ) G £ such that a G dom(/x). 
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The mapping rj : Q — > Spheres r (Y,) as required in Proposition 14.11 is provided by core. 
More precisely, we set r?(0) to be some arbitrary sphere and tj(£) = core(£) if £ 7^ 0. 

Let us come back to the example in Figured! depicting two states, £ and £', of the 
sphere automaton for radius r = 1, and a nested word that makes use of these states for 
being accepted. The sphere automaton contains a transition (£ c ,£,a,£') for some £ c . 

We will verify in the following that conditions (2)-(7) are indeed satisfied. The cases 
(3')-(7') as well as the construction of £ c are left to the reader. 

(2) All the active nodes in £' are labeled with a. 

(3) Whenever a sphere from £ is already present in £' , then the corresponding active nodes 
are in the <-relation. This applies to E\ and E[ as well as to E2 and E' 2 . 

(4) The extended sphere E' 4 is the only one in £' whose active node has no <-predecessor. 
However, the distance between this active node and the sphere center equals r = 1. 

(5) There is one extended sphere in £ without a Osuccessor wrt. the active node, namely 
£4. As required, the distance to the sphere center is r = 1. 

(6) There are three extended spheres in £' whose active nodes have a <-predecessor: E[, 
E' 2 , and E' 3 . In fact, £ contains, in terms of E%, E%, and, respectively, E3, all three 
extended spheres with the active node replaced by the respective <-predecessor. 

(7) Symmetrically to the case (6), E±, E2, and E3 from £, where the active node is followed 
by a <-successor, have their counterparts in £' in terms of E[, E' 2 , and E' 3 , respectively. 

4.2.2. Every Nested Word Is Accepted. Let W = ([n], <, Jt, A) be an arbitrary nested word 
over S. We show that W G C(B r ). Let us first distribute colors to each of the involved 
spheres. For this, we define the notion of an overlap: for any G [n], i and i' are said to 
have an r-overlap in W if r-Sph(W, i) = r-Sph(W, i') and d^(i, i') < 2r + 1. For example, 
in Figure O i and i' have a 2-overlap. 

Claim 4.4. There is a mapping x '■ \p] ~^ [4^Col\ such that, for all € [n] with i ^ i' , 
the following holds: if i and i! have an r-overlap in W, then 7^ x{^')- 

Proof. The mapping is obtained as a graph coloring. Consider the graph ([n], Arcs), Arcs C 
[n] x [n], where, for G [n], we have £ Arcs iff i 7^ i' and i and i' have an r-overlap 
in W. Observe that ([n],^4rcs) cannot be of degree greater than 4 • maxSize(r) 2 . For 
each i € [n], there are at most four distinct events i! such that d^(i,i') < 1. Now, if a 
position j E [n] wants to "get in touch" with i, it requires a position in its own sphere, 
another position in the sphere around i, and one of the four possibilities to relate these 
two positions. Hence, ([n],Arcs) can be ^Col-colored by a mapping x '■ \p] —> [#Col] (i.e., 
x(i) 7^ x(i') f° r every (i, i') £ Arcs), which concludes the proof of Claim [3~4l fj 

We now specify p : [n] — > Q: for i G [n], we set = {(r-Sph(Ty, i'), i, x{i')) I *' £ [™] 
such that d^{i,i') < r}. With this definition, we can check that, for all i G [n], is a 

valid state of B r , and that /) is indeed an accepting run of B r on W. So let i G [n] and let 
E = (iV, <, fi,X, 7, a, coZ) and £" = (N' , <', //', A', 7', a', co/') be contained in p(i). 

(a) Assume that 7 = and 7' = a'. Then, (N, <, /x, A, 7, 7) = (r-Sph(VF, i), i) and 
(N',<',fjf, \',i,i) = (r-Sph(W,i),i). Consequently, we have (N, <,p, A, 7, 7) = 
(N',<',p!, A',7',7'). Moreover, co/ = coZ' = x(z'). 

(b) Of course, A(a) = A'(a'). 
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(c) Assume (N, <, p, A, 7) = (N' , <', pf, A', 7') and coZ = col'. There are 11,12 G [n] with 
%(*,»i)j< r, <%(i,i 2 ) < r, (N, <,p,X, 7, a) (r-Sph(W , i\) , i)^(N , <,p,, X,j,o/) 
(r-Sph(W, 12), i), and col = x(h) = xfa)- Clearly, we have r-Sph(W, i\) = r-Sph(W, 12). 
Furthermore, i\ = 12 and, therefore, a = a' . This is because i\ and 12 have an r- 
overlap in W so that, according to Claim [3~4"l i\ ^ 12 would imply x(h) 7^ xfe), which 
contradicts the premise. 

Now, for i G {0, ...,n} and i' = i + 1 with i' g" dom(/I _1 ), we check that the triple 

(p(i), A(i'), p(i')) is contained in <5i, where we let p(0) = 0. Note first that, of course, 

p(i') + 0. 

(1) Suppose E = (N,<,p,,\,~/,a,col) G p(i'). We have E = (r-Sph(W, i"), i', for 
some i" G [n] with dyy(i',i") < r. As i' ^ dom(pt~ 1 ), we deduce a ^ dom(/i _1 ). 

(2) Obviously, we have label(p(i')) = A(i'). 

(3) Suppose S = (iV, <, p, A, 7, a, coZ) G (we thus have i > 1) and j £ N such 
that E 1 ^'] G Recall that we have to show that, then, (a,j) G <. There are 

G [n] such that d^(h,i) < r, d^(i[,i') < r, (iV, <, p, A, 7, a) = (r-Sph(W,ii),i), 
(N,<, p,A,7,j) = (r-Sph(W, i'), and coZ = x(h) = x(*'i)- We easily see that i\ 
and have an r-overlap in W. We deduce, according to Claim 14.41 h = i>\- As, 
then, A, 7, a) * {r-Sph(W , h) , i) , (N, <, p, A, 7, j) = (r-Sph(W, i x ), «'), and 

G <, we can infer (a, j) G <. 

(4) Let E 1 = (N,<,p,X,^,a,col) G /o(z') , suppose i' > 2, and suppose that there is no 
j G N such that (j, a) G <. Recall that we have to show that (^(7, a) = r. There 
is i\ G [n] such that d^(i'i,i') < r and (N, <,p, A, 7, a) = (r-Sph(V4 / , i^), i'). But if 
cZe(7, a) < r, then d^^^i') < r, and there must be a <-predecessor of a, which is a 
contradiction. We therefore deduce that cZ^(7, a) = r. 

(5) Let E = (N, <,p, A, 7, a, coZ) G p(z) and suppose that there is no j G N such that 
(a,j) G <. Similarly to the case (4), we show that ^(7, a) = r. In fact, there is 
i\ G [n] such that d^(ii,i) < r and (N, <, p, A, 7, a) = (r-Sph(W, ii), i). Again, if 
^(7,0;) < r, then cZ^(ii,i) < r so that there must be a Osuccessor of a, which is a 
contradiction. We conclude that ^(7, a) = r. 

(6) Let E = (N, A, 7, a, col) G yo(i') and j £ N such that (j, a) G <. We show that, 
then, E[j] G There is G [n] such that dyy(i'i,i') < r, (N, <, p, A, 7, a) = 

(r-Sph(W, ii), i'), and col = x(i'i)- As (j, a) G <, a is not minimal so that we 
have i > 1. Since, furthermore, cZ^(7,j) < r implies cZ^i'^i) < r, and since we 

also have (N, <, p, X,j, j) = (r-Sph(W , i^) , i) and col = x(^'i)) we deduce E\j] = 
(N,<,p,\,~/,j, col) G p{i). 

(7) Let £" = (N, <, A, 7, a, coZ) G and j £ N such that (a,j) G <. We have to 
show that E[j\ G /o(^')- There is i\ G [n] such that d^{i\,i) < r, (N, <, /i, A, 7, a) = 
(r-Sph(PF, ii), «), and coZ = x(h)- Since dE{^-,j) < r implies d^{i\,i') < r, and 
since we have (iV, <, /i, A, 7, j) = (r-Sph(W, ii), i') and coZ = x(*i)> we deduce ^[7] = 
(N,<,p,X,j,j, col) G p(z')- 

Next, for i c ,i,i' G [n] with il = i + 1 and (i c ,i') G ju, we check that the quadruple 
(p(i c ), p(i), A(i'), p(i')) is contained in (52. Checking (2)-(7) proceeds as in the above cases. 
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For completeness, we present the cases (3')-(7'), which are shown analogously. First observe 
that, indeed, p(i c ), p(i), and p(i') are all nonempty. 

(3') Suppose E = (JV, <, p, X, 7, a, col) G p(4) and j G JV such that E[j] G p(i')- We 
show that (a,j) G p. There are h,i'i G [n] such that dyp(ii,i c ) < r, d^{i'i,i') < r, 

(JV,<,/x,A, 7 ,a) 9* (r-Sph(W,4),4), (JV, <, p, X,-yJ) = {r-Sph(W , i[) , i') , and col = 
x(4) = Again, 4 and ^ have an r-overlap in PF. According to Claim [3~4l i\ = i\. 

Then, (JV,<,/i,A, 7 ,a) = (r-Sph(W, 4), 4), (JV, <,p,X, 7, j) ^ (r-Sph(W,*i),t'), and 
(i c ,i r ) G A*, so that we can deduce (a, j) G /x. 
(4') Let i? = (JV, <, /x, A, 7, a, coZ) € /o(i') and suppose that there is no j £ N such that 
(j, a) G /U. We have to show that ^#(7, a) = r. There is i[ G [n] such that d^(^, £') < r 

and (JV, <, /x, A, 7, a) = (r-Sph(M^, z^), i'). But if (^(7, a) < r, then d^^^i') < r, so 
there must be a /x-predecessor of a, which is a contradiction. We deduce ^(7, a) = r. 

(5') Let E 1 = (JV, <, p, A, 7, a, coZ) G p(4) and suppose that there is no j G JV such that 
(a, j) G p. We show that, then, cZe(7, a) = r. There is 4 G [n] such that cZ^(4,4) < r 
and (JV, <, p, A, 7, a) = (r-Sph(PF, zi), i c ). If ^(7, a) < r, then cf^(4, 4) < r , so there 
must be a //-successor of a, which is a contradiction. We conclude that gZ_e(7i a) = 

(6') Let E = (JV, <, p, A, 7, a, coZ) G p(i') and j & N such that (7, a) G /x. We show E\j] G 
p(4)- There is i[ G [n] such that d^(i[,i') < r, (JV, <, p, X, 7, a) = (r-Sph(W, z^), z'), 
and coZ = x(*i)- Due to ^(7,^) < r, we also have <fc?(zi,4) < r > and since 
(N,<,p,X,j,j) = (r-S-ph(W,i[),i c ) and coZ = we deduce G /o(i c ). 

(7') Let 75 = (JV, <, /x, A, 7, a, coZ) G i»(4) and j G JV such that G /x. We have to 

show E[j] G p{i'). There is 4 G [n] such that cZ^(4,4) < r, (JV, <, /x, A, 7, a) = 
(r-Sph(W, ii), z c ), and coZ = From (4;(7>j) < r, it follows d^{i\,i') < r. As, 

moreover, (JV, <, p, X, 7, j) = (r-Sph.(W, 4)j and coZ = x(4)> we deduce 75[j] = 
{N, <,p, A, 7, j, coZ) G p(z'). 

4.2.3. Every Run Keeps Track Of Spheres. We will now show that an accepting run reveals 
the sphere around any node. This constitutes the more difficult part of the correctness 
proof. 

We introduce some useful notation: By A, we denote the set {—>,<—, rv, w, xj} 
of directions. Now let W = ([n],<, p, X) G NW(S) be a nested word, i,j G [n], and let 
w = e\ . . . e m G A* (where G A for all k G {1, . . . , m}). We write % ==>w j if there are 
io, i%, . . . , i m G [n] such that io = i, i m = j, and, for every k G {0, . . . , m — 1}, one of the 
following holds: 

(a) efc+i = — > and 4+1 =4 + 1 

(b) e fc+ i = <- and =4-1 

(c) efc + i = rx and 4 G dom(p) and A(4) G and 4+1 = M( z fc) 

(d) efc + i = w and 4 S dom(/x) and A(4) G and 4+1 = ^(4) 

(e) efc + i = ^ and 4 G dom(/x _1 ) and A(4) G Sj, and 4+1 = P^iik) 

(f) efc + i = and 4 £ dom(/x _1 ) and A(4) G S^, and 4+1 = P X (4) 

If 

Moreover, we write i 1 — >w j if there are pairwise distinct io,h, ■ ■ ■ ,i m -i G [n] and i m G 

[n] \ {4, • • • , «m-l} such that iq = i, i m = j, and, for every k G {0, . . . , m — 1}, (a)-(f) as 
above hold. 
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We say that a string w G A + is circular if i c — >w i for some nested word W E NW(E) 
and some position i of W . In other words, a circular string can produce a circle in a nested 
word. For example, rx — and > w^^o— > are circular (for an appropriate alphabet 
S), whereas r>— — >^<— is not circular. 

The following proposition is crucial for our project, and it fails when considering nested 
words over more than two stacks. 

Proposition 4.5. Let w G A + be circular. Then, for all k > 2, w k is not circular. 

Before we prove Proposition 14.51 observe that it does not hold as soon as a third 
stack comes into play. To see this, consider Figure describing a part of a nested word 
W over the 3-stack call-return alphabet ({({a}, {a}), ({&}, {fr}), ({c}, {c})}, 0). Suppose 

3 3 

w = rx <— *u <— <— (where the meaning of is the expected one), which is circular 

WW 

if we apply our definition to the framework of three stacks. However, we have i c >w i- 

It should be noted that this does not imply that there is no sphere automaton or logi- 
cal characterization in the framework with more than two stacks. Indeed, we leave as an 
open question if multiple stacks generally allow for a logical characterization in terms of a 
fragment of MSO logic. 




Figure 7: Proposition 14.51 fails when considering three stacks 

w 

In the above definition of i c — >w 3i it is crucial to require the elements . . . , i m -i € 
[n] to be pairwise distinct. This can be seen considering a part of the nested word W over 
the 2-stack call-return alphabet ({({a}, {a}), ({b}, {&})}, 0) that is depicted in FigureEl Let 
w = rx^- <— <— xj <— , which is a circular string. We have i ww > w i, i.e., starting from i, 
we can follow the sequence of directions w twice, arriving at i again. However, apart from 
i, we have to visit j\ and ji twice. Indeed, % ' f > w i. 

Proof, (of Proposition gTgp. Let W = ([n], < A) G NW(S), w G A+, and i G [n]. We 

w 

have to show that, if % < — >w h then w cannot be decomposed nontrivially into identical 
circular factors, i.e., there is no circular u G A + such that w = u for some k > 20 

w 

To see this easily, we observe that a situation such as i c — >w i corresponds to a 
topological circle, as depicted in Figure [H A topological circle is a closed line in the 
two-dimensional plane that never crosses over itself. Let us construct topological circles 
according to the following procedure: We assume a straight (horizontal) line of the plane. 

^Actually, one can even show that there is no u £ A + at all (not even non-circular) such that w = u k for 
some k > 2. 
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a — ► b a — ► b b — > a b — ► a a — > o > o — ► a 




Figure 8: Intermediate positions need to be pairwise distinct 

Assume further a point i on this line. Starting from i, we choose another two points as 
follows: Pick a symbol 7 from the alphabet {^>,^,0, 0}. According to this choice, we 
first draw a semicircle above the straight line ending somewhere on the line, and then, 
without interruption, a semicircle below the line, again resulting in a point on the line. 
Each semicircle is drawn in the direction indicated by 7, e.g., Q requires to draw the upper 
semicircle rightwards and the lower one leftwards, and ~> requires both the upper and the 
lower semicircle to be drawn rightwards. This procedure is continued until we reach the 
original point i. We call a sequence from *^,0, G} + that allows us to draw a topological 
circle circular. For example, in Figure O we construct a topological circle by following the 
sequence x = ~> ~> Q starting in the left outermost point of intersection on the 

horizontal line. Thus, x is circular, whereas O O is not circular. Observe that we have 
x / y k for all y £ ^,G,G} + and fe > 2. 




Figure 9: Proof of Proposition 14.51 



It is not hard to see that topological circles behave aperiodically in general, i.e., for any 
given y 6 «-^,G, G}, there is no k > 2 such that y fe is circular. To show our proposition, 
we can even restrict to circular y. So let y € *^,Q, G} + . But if y is circular, then, for 
growing k, y k gives rise to a "spiral" , and going back to the starting point would require to 
intersect the line that has been drawn hitherto. 

Let us relate our topological circles to the nested- word setting over two stacks. To 
this aim, we define a partial mapping / : A + — » {~>,^,0, G} + that associates with any 
circular string a sequence over *^,Q, G}. This is done by reading a string from left to 
right and successively replacing every direction from A with a symbol from «^,G, G}, 
according to the following rules: 
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• r> is always replaced with 

• is always replaced with 



— > is replaced with 
<— is replaced with 
kj- is replaced with 
\j is replaced with 



I G if the previous letter has been 

I ~> otherwise 

I O if the previous letter has been rx 

I otherwise 

I G if the previous letter has been <— 

I ~» otherwise 

I G if the previous letter has been — > 

I «~<~ otherwise 



For example, /(<— ^^^j-^-^^xj) = w-O^^^GO. Let w be circular. Clearly, 
/(if) is circular as well, i.e., it allows us to draw a topological circle. We assume that the 
first letter of w stems from {r\, Other cases are either trivial or can be reduced to that 
one. Then, if w can be decomposed nontrivially into identical circular factors, then this 
also applies to f(w). Summarizing, the power of a circular string is not circular anymore. 
This concludes the proof of Proposition 14.51 □ 

We will now show that, indeed, B r discovers the r-sphere around any node of an input 
nested word. 

Let W = ([n], <, fi, A) G NW(S) be a nested word and p be a run of B r on W. Consider 
any i G [n], let (iVj, <i, fii, \i,"fi) refer to core(p(i)), and let coli be the unique element from 
[#Col] satisfying Ej := (N i} <», m, A*, 7;, 7*, co/j) G 

The following statement claims that an arbitrarily long path in E{ is simulated by a 
corresponding path in W . 

Claim 4.6. Let d > and suppose there are jo, ■ ■ ■ ,jd £ -^i such that 7^ = jo ji 
■ • • 3d- Then, there is a (unique) sequence of nodes io, . . . , i d G [n] such that 

• «o = *, 

• for each k G {0, . . . , d}, Ei[j k ] G /o(ijfe) (in particular, X(i h ) = Xi(jk)), and 

• for each k G {0, . . . , d - 1}, (jfc, jfc+i) (*fc>*Jfc+l)- 

Proof. The proof is by induction. Obviously, the statement holds for d = 0. So assume 
d > and suppose there are a sequence jo, - ■ ■ ,jd>jd+i G -^i such that 74 = jo ji 

• • • 3d jd+i and a unique sequence i Q ,ii, . . . ,i d G [n] such that z = « ; ^ib'jfc] £ /»(*) 
for each A; G {0, . . . , d}, and (jfc, jfc+i) (£jfc> *fc+i) f° r each A; G {0, . . . , d— 1}. We consider 
four cases: 

• Assume (jd,jd+i) G <j. Then, p(v) is not a final state so that id < n. We set id+i = 
Due to (7), we have #i[jrf+i] G p(id+i)- 

• Assume (jd+i,jd) G <j. Then, according to (6), id > 2. We set i d +i = id — L Due to (6), 
we also have -E^jd+i] G p(id+i). 

• Assume (Jdijd+i) G Mi- Clearly, /o(id) is a calling state so that //(id) is defined. Setting 
i d+ i = p(i d ), we have, due to (7'), Ei\j d+1 ] G p(v+i). 

• Assume (Jd+l,jd) £ Mi- According to (1), i<f G dom(/i _1 ). With (6'), letting ^+1 = 
pr x (id), we have Ei[jd+i] G /o(id+l)- 

This concludes the proof of Claim 14.61 O 



21 



B. BOLLIG 



Claim 4.7. There is a homomorphism h : r-Sph(W,i) — > core(p(i)). 
Proof. We show by induction the following statement: 

For every d G {0,... , r}, there is a homomorphism /i : d-Sph(W,i) — > 
d-Sph((A r j, <j, /Xj, Aj), such that, for each i' G [n] with d\y(i,i') < d, (*) 
we have Ei[h{i')} G p(i'). 

Of course, (*) holds for d = 0. So assume that (*) holds true for some natural number 
d G {0, . . . , r — 1}, i.e., there is a homomorphism h : d-Sph(W, i) — > d-Sph((A r j, /ij, Aj),7j) 
such that £^j[/i(i')] G /o(i') for each G [n] with dw(hi') < d. We show that then (*) holds 
for d + 1 as well. For this, let i\,i 2 G [n] such that dw (i, i\) = d and dyp(z, i 2 ) = d + 1. 

• Suppose ii < «2- Since djy(*,ii) < r -, we & lso have dE^i, h(ii)) < J". Due to (5), there 
is j'2 G such that <j j 2 . Since G we obtain, by (7) and (2), that 
\{h) = A(i 2 ) and Ei[j 2 ] G p(i 2 ). 

• Similarly, we proceed if i 2 < i\. By d^^j, h(i\)) < r and (4), there is j 2 G ATj such that 
J2 <t Since Ei[h(ii)] G we obtain, by (6) and (2), that \i{j 2 ) = A(«2) and 
Ei[j 2 ] G p(i 2 ). 

• If (ii,«2) G /i, then there exists, exploiting (5') and (7'), j 2 G A/j such that (h(ii),j 2 ) G /ij, 
Ai(j 2 ) = A(i 2 ), and £^[72] G p(i 2 )- 

• If (i 2 ,h) G /i, then we can find, due to (4') and (6'), j 2 G iVj such that (j 2 ,h(ii)) G //j, 
Ai(j2) = A(i 2 ), and Ei[j 2 ] G p(i 2 ). 

Observe that j 2 is uniquely determined by i 2 and does not depend on the choice of %\ or 
on the relation between i\ and i 2 : If we obtained distinct elements j 2 and j' 2 , then the 
constraints ^[72] G p(i 2 ) an d -Ei[j 2 ] Pfe) would imply that p(i 2 ) is not a valid state. 

The above procedure extends the domain of the homomorphism h by those elements 
whose distance to i is d + 1. I.e., for &i,Z2 G [n] with djy(i,ii) = dw(i,«2) = d + 1, 
we determined two unique elements h(i\),h{i 2 ) G ATj, respectively. Let us show that 
(*l,*2) E^ re ( p(i )) (/i(ii),/i(«2))- Suppose ii < i 2 (the case i 2 < h is symmetric). As 
G an d Si[/i(i 2 )] G p(i 2 ), we have, by (3), h(i\) <j /i(i 2 ). Similarly, with (3'), 
(«l,*2) G M implies (h(i{), h(i 2 )) G ^. □ 

Claim 4.8. There is a homomorphism h' : core(p(i)) — ► r-Sph(W, i). 

Proof. We show, again by induction, the following statement: 

For every natural number d G {0, . . . ,r}, there is a homomorphism /i' : 
d-Sph((A r j, <i,pi, Aj),7j) — > d-Sph(W, i) such that, for every j G Ni with (**) 
dsiilij) < d, we have £^[7] G p(h'(j)). 

Clearly, (**) holds for d = 0. Assume that (**) holds for some natural number d G 
{0, ... ,r — 1} and let /i' : d-Sph((Aj, <i,pi, Aj),7i) — > d-Sph(W, i) be a corresponding ho- 
momorphism. Let ji,j2 G Ni such that de i (7i,ji) = d and dE l {^i,32) =d+ 1. 

Suppose that j\ <» j 2 . As -£i[ii] e p{h'{ji))-, p{h'{ji)) cannot be a final state of B r so 
that there is i 2 G [n] such that h'(ji) < i 2 . Clearly, we have ^[72] G p(i%). Analogously, 
we proceed in the cases j 2 <j ji, (ji,j 2 ) G yUj, and (j 2 ,ji) G /ij to obtain such an element 
i 2 . Note that £2 is uniquely determined by j 2 and does not depend on the choice of j\ or on 
the specific relation between j\ and j 2 . This is less obvious than the corresponding fact in 
the proof of Claim [4~7l but can be shown along the lines of the following procedure, proving 
that the extension of the domain of h! by elements j G ATj with d^; i (7j, j) = d + 1 is a 
homomorphism: 



ON THE EXPRESSIVE POWER OF 2-STACK VISIBLY PUSHDOWN AUTOMATA 



25 



mod(k) 



We show that, for j,f G N{ with d Ei (^i,j) = rfs 4 (7i ; j') = d + 1, we have (j,/) C^; 
(h'(j),h'(j')) (where the elements h'(j) and /i'(j') are obtained as indicated above). So 
suppose j j' ■ There are £ G {0,...,d} and pairwise distinct jo, ■ ■ ■ ,j2(d+l)-l G ^ii 
such that 

je+i ■■■ ^E, jd+i = j 

li = Jo <— > Ei • • • ^E % 31 £ i 

J2(d+1)-I ■ • • = f 

For ease of notation, set D = 2{d + 1) — £ and let, for k G IN, 

fc if k < D 

((k - £) mod (D - £ + 1)) + £ if k> D 

I.e., the mapping mod counts until D and afterwards modulo D — t + 1. According to 
Claim l4~6l there is a unique infinite sequence io,h, ■ ■ ■ G [n] such that 

• i = i, 

• for each k G IN, Ei\j mod ^] G and 
for each k G IN, (j mo d(k), 3mod(k+i)) (*fc>*fc+i)- 

In what follows, we show that ^d+i = which implies (jd+i, jd+2) (*d+ij*d+2) so that 
(jd+iJd+2) ( h 'Ud+i), h'{j d+ 2)). There is a circular string 10 = e t . . . e D G A+ such that 

w 

• Jl = ^E i 31, 

• jl ee - et+k - 1 > E ^ j g+k f or eac h A; G {1, . . . , D - £}, and 

• V ==^W ii+k(D-e+i) for each fe > 1. 

We can obtain such a w by setting, for each k G {£,... , D}, 

—> if jk <i 3mod(k+l) 
*— ^ 3mod(k+l) <i 3k 

r\ if Xi(jk) G and (jk, 3mod(k+i)) G Mi and j fc ^ j m0( i(fc+i) 

^ if Aj(ifc) G and (j mo d(k+i), jk) € ^ and J mod ( fc+ i) & Jfc 

^ if Ai(ifc) G £;? and (JkJmod{k+l)) G Mi and j fc & j mo d(fc+i) 

if Aj(ife) G 5^ and (j mod (k+i), jk) G ^ and j mod (fc+i) & jfc 

As [n] is a finite se10, there are p, q G IN such that £ < p < q and i p = i q . We choose 
p and g such that it,... ,iq-i are pairwise distinct. We have both Ei\j mod ^] G p(i p ) and 
^ibmod(g)] G p(i p )- According to the definition of the set of states of B r , this implies 
jmod( P ) = jmod(q)- Let us distinguish three cases: 

Case 1: p = £ and q = £ + — £ + 1) for some k > 1. Then, ^ c >w H+k(D-i+i) so 

that, according to Proposition 14.51 we have fc = 1 and ^ = 273+1, and we are done. 



efe 



^In the context of infinite nested words, this argument can be replaced with the fact that, starting in i, 
there is no infinite sequence of pairwise distinct nodes that follows the infinite sequence of directions vf , 
i.e., the infinite repetition of w (see Section [BJ. 
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Case 2: p > I and q = p + k(D — i + 1) for some k > 1. Setting e = e mo d( p _i), we have 

g g 

both ip_i 1 — >vf %> and 5 — >w ip, which is a contradiction, as i p -\ ^ i q -i- 
Case 3: p > t and q ^ p + — £ + 1) for every k > 1. But this implies mod{p) ^ mod{q) 

and, as the j^, ■ ■ ■ ,jD are pairwise distinct, j mo d(p) 7^ 3mod(q)i a contradiction. 
This concludes the proof of Claim 14,81 □ 

So let /i : r-Sph(W, i) — » core(p(i)) and /i' : core(p(i)) — > r-Sph(W,i) be the unique 
homomorphisms that we obtain following the constructive proofs of Claims 14.71 and 14.81 
respectively. It is now immediate that h is injective, h~ l = h' , and h : r-Sph(W,i) — > 
core{p{i)) is an isomorphism. 

Recall that r/ : Q ^ Spheres r (S) shall map the empty set to an arbitrary sphere 
and a nonempty set £ £ Q onto core(£). Indeed, we constructed a generalized 2nwa 
$r = 5, Qi-,F, C) together with a mapping r] : Q — > Spheres r (E) such that 

• C{B r ) is the set of all nested words over S (cf. Section l4.2.2p . and 

• for every nested word W G NW(S), every accepting run p of B r on W, and every node i 
of W, we have rj(p{i)) = r-Sph(P^,i) (cf. Section . 

This shows Proposition 14. 1[ 

5. Grids and Monadic Second-Order Quantifier Alternation 

In this section, we show that the monadic second-order quantifier-alternation hierarchy 
over nested words is infinite. In other words, the more alternation of second-order quantifi- 
cation we allow, the more expressive formulas become. From this, we can finally deduce that 
2-stack visibly pushdown automata cannot be complemented in general. In the proof, we 
use results that have been gained in the setting of grids. By means of first-order reductions 
from grids into nested words, we can indeed transfer expressiveness results for grids to the 
nested- word setting. Let us first state a general result from [17], starting with the formal 
definition of a strong first-order reduction: 

Definition 5.1 ( [17], Definition 32). Let C and C be classes of structures over relational 
signatures r and r', respectively. A strong first-order reduction from C to C with rank 
m > 1 is an injective mapping $ : C — ► C such that the following hold: 

(1) For every G € C, the universe of the structure $(G) is Ufce{i m }({^} x dom(G)), i.e., 
the disjoint union of m copies of dom(G), where dom(G) shall denote the universe of 
G. 

(2) There exists some ip(xi, . . . ,x m ) £ FO(r') such that, for every structure G £ C, every 
ui,...,u m G dom(G), and every k 1 ,...,k m G [m], \= ip[(ki,ui), . . . , (k m ,u m )] iff 
((k±,ui), . . . , (k m ,u m )) = ((1, u\), . . . , (m, ui)). (The intuition is that u G dom(G) is 
represented by a model ((1, u), . . . , (m, it)) of ijj.) 

(3) For every relation symbol r' from r', say with arity I, and every k : [I] — ► [m], there 
is ip r K (x±, . . . , xi) G FO(t) such that, for each G G C and each u\, . . . , Ui G dom(G), 
G h rtlui, ...,«,] iff h r'Ml), Ul ), 

(4) For every relation symbol r from r, say with arity /, there is ip r (xi, ...,xi) G FO(r') 
such that, for each G G C and each u\, . . . ,ui G dom(G), G \= r[u±, . . . , ui] iff |= 

99 r [(l,Ul), . . . , (l,Ul)]. 
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Once we have a strong first-order reduction from C to C, logical definability carries over 
from C to C: 

Theorem 5.2 ( [17], Theorem 33). Let C and C be classes of structures over relational 
signatures r and t' , respectively. Let $ : C — > C be a strong first- order reduction such that 
<3?(C) is Si(r')- definable relative to C . Then, for every C C C and k > 1, C is Sfc(r)- 
definable relative to C iff&(£) is S ki T ')~ definable relative to C. 

We proceed as follows. We first recall the notion of the class of grids, of which we know 
that the monadic second-order quantifier-alternation hierarchy is infinite. Then, we give a 
strong first-order reduction from the class of grids to the class of nested words over a simple 
2-stack visibly pushdown alphabet so that we can deduce that the monadic second-order 
quantifier-alternation hierarchy over nested words is infinite, too. Note that we will add to 
ordinary grids some particular labeling in terms of a and 6, which will simplify the upcoming 
constructions. It is, however, easy to see that well-known results concerning ordinary grids 
extend to these extended grids (cf . Theorem 15.31 below) . 

We fix a signature Tends = \Pa-,Pbi succi, SUCC2} with P a ,Pb unary and succi,succ2 
binary relation symbols. Let 77, tyi ^ 1 be natural numbers. The (77*, rri}-grid is the TQ r id s - 
structure G(n, m) = ([n] x [777], succi, SUCC2, P a , Pb) such that succi = {((7, j), (i + 1, j)) \ i 6 
[n-l],je H}, succ 2 = {((«, j), + I i G [n], j G [777- 1]}, P a = {(i,j) G [77] x [m] | j 
is odd}, and Pb = G [77] x [777] | j is even}. The (3, 4)-grid is illustrated in Figure [TUl 

By G, we denote the set of all the grids. 

a — ► b — ► a — ► b 

I I 1 1 

a — ► b — ► a — ► b 

i i i i 

a — ► b — ► a — ► b 

Figure 10: The (3,4)-grid 



Theorem 5.3 ( [17]). The monadic second-order quantifier- alternation hierarchy over grids 
is infinite. I.e., for every k > 1, there is a set of grids that is Sfc + i(r 'Grids) -definable relative 
to G but not '£k(TGrids)-definable relative to G. 

For the rest of this section, we suppose that S is the 2-stack call-return alphabet given 
by = {o}, = {a}, T? c = {b}, = {b}, and = 0. In particular, the following 
results assume all alphabets apart from Ej n i to be nonempty. 

We now describe an encoding : G — > NW(S) of grids into nested words over S. Given 
77, 777 > 1, we let 



nested (a 11 [{ab) n {ba) n ] {m 1)/2 tf) if m is odd 
nested (a n [(a6) n (6a) n ] m/2 ~ 1 (a6) n 6 n ) if m is even 



<I>(G(77 , 777)) : = 

The idea is that the first 77 a's (and, as explained below, the corresponding return events) in 
a nested word represent the first column of G(n, m) seen from top to bottom; the first 77 6's 
represent the second column, where the column is seen from bottom to top; the second 77 a's 
stand for the third column, again considered from top to bottom, and so on. The encoding 
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Figure 11: The encoding $((7(3,4)) of the (3,4)-grid as a nested word 

<J>(Cr(3,4)) of the (3,4)-grid as a nested word is depicted in Figure [TT1 We claim that <3? is 
indeed a strong first-order reduction from the set of grids to the set NW(S) of nested words 
over X. Observe that <&(G(n, to)) does not have as domain the set {1,2} x ([n] x [to]) as 
required in the definition of a strong first-order reduction. However, below, we will introduce 
a bijection x n m '■ {1; 2} x ([n] x [to]) — > [2 ■ n ■ m] to identify every element in the domain 
of &(G(n,m)) with some element in {1,2} x ([n] x [to]). 

Proposition 5.4. We have that <3> : G — > NW(S) is a strong first-order reduction with rank 
2. Moreover, $(G) is Si(rg)- definable relative to NW(S). 

Proof. Let us first introduce a useful notation. Given a nested word W = ([n], <, A) and 
c € £ such that W contains at least k positions labeled with c, we let pos c (W,k) denote 
the least position i in W such that \{j G [i] | A(j) = c}| = (i.e., pos c (W, fe) denotes the 
position of the k-th c in W). 

Let n, m > 1 and let ([2 • n • to], < , /i, A) refer to $(G(n, to)). Recall that A can be seen 
as the collection of unary relations A c = {i G [2 ■ n ■ m] \ X(i) = c} for c € S. Let us map 
any node in the (n, m)-grid (i.e., any element from [n] x [to]) to a position of $(G(n, to)) 
by defining a function Xn,m '■ [ n ] x [m] — > [2 • n ■ to] as follows: 



pos a (&(G(n,m)),n ■ [(j + l)/2 - 1] + i) if j is odd 

pos b ($>(G(n, to)), n • [j/2 — 1] + (n + 1 — i)) if j is even 

for any (i,j) € [n] x [to]. Intuitively, Xn,m(i,j) G [2 • n • m] represents the node (i, j) in 
the (n, ?7i)-grid. This mapping is further extended towards a bijection x n m : {1, 2} x ([n] x 
[to]) — > [2 • n • to] as required by Definition 15.11 (item (1)). Namely, we map Xnm(lj (hj)) 
onto Xn,m(i,j) and x„, m (2, (i, j)) onto ^(Xn,m(hj))- 

We are prepared to specify the first-order formulas as supposed in Definition 15.11 Let 

ip(x 1 ,x 2 ) = n(x 1 ,x 2 ) ■ (2) 

Indeed, for every n, m > 1, fei, k 2 G {1, 2}, and u\,u 2 G [n] x [to], we have 

$(G(n,m)) h ^[Xn.m^i^iJjXji.mC^)^)] iff (( fc l^l)> ( fc 2,«2)) = ((l,wi), (2,«i)) • 

We will identify a map k : [Z] — > {1, 2} with the tuple (k(1), . . . , k(/)). Let, for c G £ and 
KG {1,2}, 

P c {x) if c G {a, 6} and k = 1 
V^ c (aO = "j Pc(a;) if c G {a, 6} and « = 2 (3) 
false otherwise 
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if K= (1,1) 

if k = (2, 2) 

if K= (1,2) 

if K= (2,1) 



where we let a = a and b = b. For every n, m > 1, n G {1, 2}, and -u € [n] x [m], we have 

G(n,m) \= ^(x)[u] iff d>(G(n,m)) |= (A(x) = c)^ n>m («,«)] . 
Further, let, for k G {1,2} x {1,2}, 

SUCCl(xi,X 2 ) A Sz SUCC2(z,Xl) 

(Pa{x\) A SUCCi(x2,^l) A SUCC 2 (xi,z) \ 

V Pb{x\) A succi(xi, X2) A succ2(xi, 2) y 
^ (xi = X2) A -Pa(^l) A -i3z succi(xi, z) \ 
<Pk (xi,x 2 ) = { V (xi = x 2 ) A Pi(xi) A -i3z succi(z,xi) 

V P a (^i) A ^(^2) A 3z (succi(z,xi) A succ 2 (z,x 2 )) 
y V Pfe(xi) A P a (x 2 ) A 3z (succi(z, xi) A succ 2 (x 2 ,z)) y 
/ P a (xi) A P&(x 2 ) A succ 2 (xi,x 2 ) \ 
\ V P b (xi) A P a {x 2 ) A succ 2 (xi,x 2 ) y 
For every n, m > 1, /c G {1,2} x {1,2}, and u\,u 2 G [n] x [m], we have 

G(n,m) |= ^(xi,x 2 )[«i,« 2 ] iff $(G(n,m)) |= (x : < x 2 )[Xn,m(' e ( 1 )> X n , m («( 2 )i u s)] • 
Finally, to complete step (3), let, for k G {1, 2} x {1, 2}, 

f xi = x 2 if «= (1,2) 
/a/se otherwise 
Then, for every n, m > 1, k G {1, 2} x {1, 2} and ui, u 2 G [n] x [m], 

G(n,m) \= ^(xi,x 2 )[ni,u 2 ] iff $(G(n,m)) |= (/i(xi,x 2 ))[Xn,m(«( 1 )> u i).Xn,m(«( 2 )> u 2)] ■ 
Let 

¥> Po (x) = (A(x) = a) and ip Pb (x) = (A(x) = 6) . (4) 
Of course, we have, for each n, m > 1, c G {a, 6}, and u G [n] x [m], 

G(n,m) |=P c (x)[u] iff $(G(n,m)) |= (</ c )[x n , m (l, «)] . 

Let 

/ A(xi) = a A A(x 2 ) = a A (xi < x 2 V 3z (xi < z A z < x 2 )) \ 

V? SUCCl (xi,x 2 ) = 

\ V A(xi) = b A A(x 2 ) = 6 A 3z (x 2 < z A z < xi) J 

and let furthermore 

V? SUCC2 (xi , x 2 ) = 3z (fi(x 1 ,z) A z < x 2 ) . 
Then, for each n, m > 1, ui,u 2 G [n] x [m], and k G {1,2}, it holds 

G(n,m) |=succ fc (xi,S2)[«i,«2] iff $(G(n,ro)) |= ((^ SUCCfc )[x n , m (l,ni),x„, m (l,^2)] • 

With the above formulas, it is now immediate to verify that $ is indeed a strong first-order 
reduction. 

Now observe that 3>(G) is the "conjunction" of 



+ a 



• the regular expression ya + [(afo) + (6a) + ] * a 

• the first-order formula Vx3y (/x(x, y) V fi(y, x)) , and 



■[(a6)+(6a)+]*(a6)+6+), 
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• the first-order property (written in shorthand) 

Vx 1 ,x 2 ,2/i,2/2 ( A(xi) = A(a; 2 ) A fj,(xi,yi) A ^(a; 2 ,y 2 ) 

-> (A(xi) = a A x 2 - xi = 1 -> yi-y 2 G{l,2}) 

A (A(yi) = a A yi - y 2 = 1 -> x 2 - x\ G {1, 2}) 
A (A(yi) = b A y x - y 2 = 1 -> x 2 - an = 2) 
A (x 2 ~ x x = 2 A A(x! + 1) ± X(xx) -» yi - y 2 G {1, 2}) 
A (yi - y 2 = 2 A A(y 2 + 1) + A(y 2 ) -» x 2 - a* G {1, 2})) 

As the regular expression represents a Si(rg)-definable property, $(G) is Xli(rg)-definable 
relative to NW(X), which concludes the proof of Proposition 15.41 □ 

Combining Theorem 15.21 Theorem 15.31 an d Proposition 15.41 we obtain the following: 

Theorem 5.5. The monadic second- order quantifier- alternation hierarchy over nested words 
is infinite. I.e., for all k > 1, there is a set of nested words over X (with X as specified 
above) that is £fc + i(rg) -definable relative to NW(S) but not Sfc(rg)- definable relative to 
NW(X). 

Recall that Theorem 15.51 relies on a particularly simple call-return alphabet and the 
presence of at least two stacks. Indeed, its proof is based on the 2-stack call-return alphabet 
X, which is given by Xj = {a}, X* = {a}, Y? c = {&}, Y% = {b}, and T, int = 0. 

Finally, Theorems 14.31 and 15.51 give rise to the following theorem: 

Theorem 5.6. The class of nested-word languages that are recognized by 2vpa is, in gen- 
eral, not closed under complementation. More precisely, there is a set C of nested words 
over X (with X as specified above) such that the following hold: 

(1) There is a 2vpa A over X such that C{A) = C 

(2) There is no 2vpa A over X such that C(A) = NW(X) \ £. 

This implies that the deterministic model of a 2vpa (see [13] for its formal definition) 
is strictly weaker than the general model. This fact was, however, already shown in [13]: 
Consider the language L = {{ab) m c n d m ~ n x n y m ~ n \ m G M, n G [m]} and the 2-stack call- 
return alphabet X given by X* = {a}, X* = {c, d}, Y? c = {&}, Xy = {x,y}, and Ej n t = 0. 
Then, L is accepted by some 2vpa over X but not by any deterministic 2vpa over X. 
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6. Buchi Multi-Stack Visibly Pushdown Automata 

We now transfer some fundamental notions and results from the finite case into the 
setting of infinite (nested) words. 

6.1. Biichi Multi-Stack Visibly Pushdown Automata. Let K > 1, and let E = 

^r)}se[K]j ^int) be a if -stack call-return alphabet. 

Definition 6.1. A Biichi multi-stack visibly pushdown automaton (Biichi Mvpa) over E is 
a tuple A = (Q, F, 6, Qi,F) whose components agree with those of an ordinary Mvpa, i.e., 
Q is its finite set of states, Qi C Q is the set of initial states, F C Q is the set of final states, 
F is the finite stack alphabet containing the special symbol _L, and 6 is a triple (8 c ,5 r ,5i n t) 
with <5 C C Q x E c x (r \ {-L}) x Q, S r C Q x E r x T x Q, and <5j„j CQx Sj nt x Q. 

A Biichi 2-stack visibly pushdown automaton (Biichi 2vpa) is a Biichi MVPA that is 
defined over a 2-stack alphabet. 

Consider an infinite string w = a\a2 . . . € A run of the Biichi Mvpa A on w 

is a sequence p = (qo, erg, . . . , a K )(q±, a\, . . . , af) . . . G (Q x Corai^) (recall that Cont = 
(r \ {_L})* •{_!_}) such that qo G Q/, ctq = _L for every stack s G [if], and [Push], [Pop], and 
[Internal] as specified in the finite case hold for every i G M + . We call the run accepting if 
{Q \ Q = Qi f° r infinitely many % G M} flF/l. A string G S w is accepted by if there is 
an accepting run of A on w. The such defined (string) language of A is denoted by L U (A). 

For the infinite case, we can likewise establish a relational structure of infinite nested 
words: 

Definition 6.2. An infinite nested word over S is a structure (IN + ,<,/z, A) where < = 
{(i,i + 1) | i G M + }, A : M + — > S, and = U se [x] M s ^ w + x 1N+ where, for every s G [if] 
and (i,j) G M + x M + , (i,j) G fi s iff i < j, A(i) G EJ, A(j) G EJ, and A(i + 1) . . . A(j - 1) is 
s-well formed. 

The set of infinite nested words over E is denoted by NW J (S). Again, given infinite 
nested words W = (M + , < , fx, A) and W = (M + , <', //', A'), A = A' implies VP = W so that 
we can represent W as string(W) := A(1)A(2) . . . G E w . Vice versa, given a string G E w , 
there is exactly one infinite nested word W over E such that string (VP) = w, which we 
denote nested(w;). 

Definition 6.3. A generalized Biichi multi-stack nested-word automaton (generalized Biichi 
Mnwa) over E is a tuple B = (Q,5,Qj,F,C) where Q, 5, Qj, F, and C are as in a 
generalized Mnwa. Recall that, in particular, 5 is a pair (61,62) with 5± C Q x E x Q and 
62 Q Q x Q x E r x Q. 

We call a generalized Biichi 2-stack nested-word automaton (generalized Biichi 2nwa) 
if it is defined over a 2-stack alphabet. 

If C = 0, then we call B a Biichi Mnwa (Biichi 2NWA, if K = 2). 

A run of B on an infinite nested word VP = (1N + , <,//, A) G NW^(E) is a mapping 
p : M + — > Q such that (<j, A(l), p(l)) G 5± for some g G (J/, and, for all i > 2, we have 

(p(p^ 1 (i)) , p(i — 1) , X(i) , p(i)) G #2 if A(i) G E r and p~ l (i) is defined 
(p(i — 1), \(i), p(i)) G <5i otherwise 



32 



B. BOLLIG 



The run p is accepting if p(i) G F for infinitely many i G 1N+ and, for all i G M+ with 
p(i) G C, both A(i) G S c and is defined. The language of B, denoted by C^(B), is the 
set of infinite nested words over £ that allow for an accepting run of B. 

As we still have a one-to-one correspondence between strings and nested words, we may 
let C U (A) with A a Biichi Mvpa stand for the set {nested(w) | w G L UJ (A)}. 

It is now straightforward to adapt Lemma 12.71 and Lemma 12.81 to the infinite setting: 

Lemma 6.4. For every generalized Biichi Mnwa B, there is a Biichi Mnwa B' such that 
C U (B') = C W {B). 

Lemma 6.5. Let C C NW W (S). The following are equivalent: 

(1) There is a Biichi Mvpa A such that C W {A) = C. 

(2) There is a Biichi Mnwa B such that C U (B) = C. 

6.2. Biichi 2-Stack Visibly Pushdown Automata vs. Logic. In this section, we will 
again restrict to two stacks. Unfortunately, EMSO logic over infinite nested words turns 
out to be too weak to capture all the behaviors of Biichi 2vpa. Given that EMSO logic 
considers a successor relation instead of an order relation, one cannot even express that one 
particular action occurs infinitely often. To overcome this deficiency, one can introduce a 
first-order quantifier 3°°xp meaning that there are infinitely many positions x to satisfy the 
property ip [4]. 

So let us fix a 2-stack call-return alphabet £ = ({(£;:, Ej), (£;?, Xy)}, Sjnt) for the rest 
of the paper. We introduce the logic MS0 oo (rg), which is given by the following grammar: 

ip ::= \(x) = a \ x < y \ p{x,y) \ x = y \ x G X 

—up | (pi V ip2 | 3xip | z\°°xip | 3Xip 

where a € S. The fragments EMSO°°(rg) and FO°°(Tg) are defined as one would expect. 
The satisfaction relation is as usual concerning the familiar fragment MSO(Tg). Moreover, 
given a formula ipijj, x\, . . . , x^^ X 1 ,...,X n ) G MSO°°(rg), an infinite nested word W, 

(h,...,i m ) G (IN+r, and (h,...,I n ) G (2 M +)", we set W \= (3°°^)[ii, . . . ,i m , h, ■ ■ ■ , In] 
iff W |= ip[i, ii, . . . , i m , I\, . . . , I n ] for infinitely many i G W + . Given a sentence ip G 
MSO°°(rg), we denote by the set of infinite nested words over S that satisfy (p. 

To establish a connection between the extended logic and our Biichi automata models, 
we have to provide an extension of Hanf 's Theorem. 

Theorem 6.6 (cf. [4]). Let ip G FO°°(rg) be a sentence. There is a positive Boolean 
combination ip of formulas of the form 

3 _< xx(x) and 3 >i: x x( x ) and3 <oc xx( x ) and3~°°xx(x) 

where t G IN and x( x ) G FO(rg) is local such that, for every nested word W G NW W (S), we 
have 

W^piffW^^p. 

Unfortunately, we do not know if ip can be computed effectively in this extended setting. 
We observe that the 2nwa B r constructed in the proof of Proposition 14.11 can be easily 
adapted to obtain its counterpart for infinite nested words: 

Proposition 6.7. Let r G IN be any natural number. There are a generalized Biichi 2nwa 
By = (Q,5,Qj,F,C) over S and a mapping r) : Q — > Spheres r (Ti) such that 
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• iy(B^) = NW U (E) and 

• for every W G NW W (S), every accepting run p of B% on W, and every node i G 1N + of 
W , we have rj(p(i)) = r-Sph(W,i). 

Proof. First, note that Proposition 14.51 and the crucial argument stated in the proof of 
Claim 14,81 (see Footnote [5]) hold for infinite nested words just as well. Now, we look at 
the generalized 2NWA B r = (Q,S,Qi, F,C) as constructed in the proof of Proposition 14.11 
As the only purpose of the set F of final states is to ensure progress in some states where 
progress is required in terms of spheres with a non-maximal active node, we can set B% to 
be (Q, 5, Qi, Q, C), and we are done. □ 

With this, we can easily extend Lemma 14.21 and determine a Biichi 2nwa to detect if a 
particular sphere occurs infinitely often in an infinite nested word: 

Lemma 6.8. Let r £ H and let S G Spheres r (S) . There is a generalized Biichi 2NWA B 
over S such that £ w (i3) = {W G NW aJ (S) | there are infinitely many i G JN+ such that 
r-Sph(W,i) ^ S}. 

Proof. We start from the generalized Biichi 2nwa B% = (Q,6,Qj,Q,C) and the mapping 
7] : Q — > Spheres r (E) from Proposition 16.71 To obtain B as required in the proposition, we 
simply set the set of final states to be {q G Q | 77(g) = S}. □ 

Theorem 6.9. Let C C NW aj (S) be a set of infinite nested words over the 2-stack call- 
return alphabet S. Then, the following are equivalent: 

(1) There is a Biichi 2vpa A over S such that C W (A) = C. 

(2) There is a sentence <p G EMSO°°(rg) such that ^(cp) = C. 

Proof. To prove (1) — > (2), one again uses standard methods. Basically, second-order 
variables X q for q G Q encode an assignment of states to positions in a nested word. Then, 
the first-order part of the formula expresses that this assignment is actually an accepting run. 
To take care of the acceptance condition, we add the disjunction of formulas 3 = °°x (x G X q ) 
with q a final state. 

For the direction (2) — > (1), we make use of Lemmas 16. 4( 16.51 16. 8| (a simple variation 
of) Lemma l4.2( and the easy fact that the class of languages of infinite nested words that 
are recognized by generalized Biichi 2nwa is closed under union and intersection. With 
this, the proof proceeds exactly as in the finite case. □ 



7. Open Problems 

We leave open if visibly pushdown automata still admit a logical characterization in 
terms of EMSO logic once they are equipped with more than two stacks. 

We conjecture that every first-order definable set of nested words over two stacks is 
recognized by some unambiguous 2vpa, i.e., by a 2vpa in which an accepting run is unique. 
To achieve such an automaton, the coloring of spheres as performed by B r by simply guessing 
and subsequently verifying it has to be done unambiguously. 

We do not know if EMSO logic over nested words becomes more expressive if we allow 
atomic formulas x < y with the obvious meaning. For this logic, it is no longer possible to 
apply Hanf 's theorem as the degree of the resulting structures is not bounded anymore. 
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Our method might lead to logical characterizations for concurrent queue systems, where 
several visibly pushdown automata communicate with each other via channels [14]. In this 
extended setting, we deal with both multiple stacks and channels. A corresponding logic 
then has to provide an additional matching predicate msg(x, y) to relate the sending and 
reception of a message (see, for example, [5]). It remains to identify channel architectures 
for which a logical characterization is possible. Using results from [14], this might lead to 
partial results concerning the decidability of corresponding satisfiability problems. 

Finally, it might be worthwhile to study if our technique leads to a logical characteri- 
zation of 2VPA for more general 2-stack call-return alphabets as introduced in [8]. 
Acknowledgment We thank the anonymous referees for their careful reading and many 
useful remarks. 
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